Maybe it'll speed up our CI?
https://github.com/Swatinem/rust-cache

My investigation log

Adding caching

First run (with no cache present) is looking real bad:

• windows test +1m 8s
• windows difftest +6m 24s
  • no idea why the cache would take so long when there's nothing to download and nothing has been compiled???
• all "install rust toolchain" are now 0s, so we probably have to move them before the cache action

First CI with restore

Comparing latest main to this PR:

• overall time: 27m 39s -> 16m 8s
• longest job "windows-difftest": 27m 29s -> 16m 0s
• cache restore, measured from seconded empty commit in this PR, which may have been removed:
  • windows: 58s to 2m 3s
    • first run of windows difftest has an obscene "restore" of 6m 24s even though there's nothing to restore???
  • linux: ~30s
  • mac: 30s - 40s
• cache creation, measured from first commit in this PR:
  • windows: 30s - 1m
  • linux: 8s - 15s
  • mac: 30s - 50s

Install toolchain before restore (run attempt #1)

• rust-cache seems to install the toolchain for you via rustup if you don't do it yourself, toolchain install has 0s execution time
• Action: move it before cache restore
• Results: old rust-cache time is now split between rustup install and new rust-cache
• Could caches contain rustup builds? Try purging caches

purging caches and retrying CI (run attempt #2)

• as expected, a lot of runtime is moved from "rust-cache" to "install toolchain"
• rust-cache restore times:
  • windows: 2-3s
  • mac: 1-2s
  • linux: 1-2s
  • => couldn't repo crazy CI times from first attempt, just a fluke?
• cache creation:
  • windows: 30-50s
  • mac: 30-40s, 60s fluke?
  • linux: ~10s
• Results: rustup before rust-cache to separate out toolchain install time

Lessen cache usage

• github cache is reporting high cache usage
  • 10GB limit by gh
  • caches take up:

• Idea: can we reuse the same cache between actions?
  • only test creates the cache, for each platform
  • difftest may get a separate cache for the difftest target dir
• attempt #1, creating caches
  • windows randomly taking 15min to rustup install toolchain, likely the culprit of rust-cache taking so long the first time?
• attempt #2, restoring caches
  • Mac: vulkan sdk caching doesn't work at all, but still creates new caches? Cache restore on MacOS broken jakoch/install-vulkan-sdk-action#558
  • jobs have different cache hash due to rustup env vars... retry entire experiment

remove rustup env vars

• cache usage (limit of 10GB):

• CI times:
  • own cache @ de88135
  • test cache @ 7519e71
  • main @ 79acad1

• Analysis of difftest slowdown:
  • windows & mac +6min:
    • +3min from "test difftest" aka. cargo test -p difftest
    • +3min from "difftest" themselves
  • linux follows similar pattern but less worse

move "test difftest" to test job

• Idea: On main windows difftest:
  • test difftest: 7m 29s
  • difftest: 14m 44s
  • move work from the longest running task difftest to test
  • maybe also improve rust-cache reuse, and speed up difftest some more?